Image capturing apparatus, control method therefor, and storage medium

ABSTRACT

An image capturing apparatus comprises an image capturing device, a detection unit configured to detect the subject from the image captured by the image capturing device, a setting unit configured to set a plurality of focus detection frames inside the image based on a result of the detection by the detection unit, a focus detection unit configured to detect a focus state and reliability thereof in each of the plurality of focus detection frames; and a selection unit configured to select a main focus detection frame for performing focus adjustment based on results of the detections by the detection unit and the focus detection unit, wherein the selection unit is configured to cause a method of selecting the main focus detection frame to vary depending on a part of the subject detected by the detection unit.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a focus adjustment technique in animage capturing apparatus.

Description of the Related Art

In recent years, a variety of AF methods (autofocus methods) that use animage sensor, such as an image capturing surface phase-difference AFmethod and a contrast AF method, have been put into practical use. Also,for such AF methods, a technique to achieve focus by identifying aregion of a main subject has been established.

As this method, Japanese Patent Laid-Open No. 2010-191073 performscontrol to detect neighboring blocks that fall within a predetermineddepth from among a plurality of AF frames, and select a main AF framefrom among the blocks.

Also, Japanese Patent Laid-Open No. 2015-041901 improves the precisionof identification of a main subject region by using color information inaddition to detection of blocks that fall within a predetermined depth.

Furthermore, Japanese Patent Laid-Open No. 2019-121860 discloses amethod of detecting a pupil and the like as body parts included in aface, judging the reliability degrees of the detection results inrelation to the detected body parts, and setting a focus detectionregion in a region with a high detection reliability degree. In thisway, focusing can be performed in a state where a main subject positionhas been identified with higher precision.

However, as the techniques described in Japanese Patent Laid-Open No.2010-191073, Japanese Patent Laid-Open No. 2015-041901, and JapanesePatent Laid-Open No. 2019-121860 detect a region of a main subject withhigh precision and set a focus detection area based on the detectionresult, they do not always perform focus detection in a regionappropriate for focus detection. If a region that was judged to be amain subject is not a region appropriate for focus detection, accuratefocus adjustment may not be able to be performed.

SUMMARY OF THE INVENTION

The present invention has been made in view of the aforementionedproblem, and performs focus adjustment accurately in a region with ahigher priority degree while avoiding regions in which focus detectionis difficult.

According to a first aspect of the present invention, there is providedan image capturing apparatus, comprising: an image capturing deviceconfigured to capture an image of a subject; at least one processor orcircuit configured to function as a detection unit configured to detectthe subject from the image captured by the image capturing device; asetting unit configured to set a plurality of focus detection framesinside the image based on a result of the detection by the detectionunit; a focus detection unit configured to detect a focus state andreliability thereof in each of the plurality of focus detection frames;and a selection unit configured to select a main focus detection framefor performing focus adjustment based on results of the detections bythe detection unit and the focus detection unit, wherein the selectionunit is configured to cause a method of selecting the main focusdetection frame to vary depending on a part of the subject detected bythe detection unit.

According to a second aspect of the present invention, there is provideda control method for an image capturing apparatus that includes imagecapturing device for capturing an image of a subject, the control methodcomprising: detecting the subject from the image captured by the imagecapturing device; setting a plurality of focus detection frames insidethe image based on a result of the detection in the detecting;performing focus detection to detect a focus state and reliabilitythereof in each of the plurality of focus detection frames; andselecting a main focus detection frame for performing focus adjustmentbased on results of the detections in the detecting and the focusdetection, wherein the selecting causes a method of selecting the mainfocus detection frame to vary depending on a part of the subjectdetected in the detecting.

Further features of the present invention will become apparent from thefollowing description of embodiments with reference to the attacheddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of a digital camera,which is one embodiment of an image capturing apparatus of the presentinvention.

FIG. 2 is a flowchart illustrating the operations of the digital cameraof one embodiment.

FIG. 3 is a flowchart for describing the operations for setting AFframes.

FIGS. 4A and 4B are diagrams showing the concept of a detected regionregarding a face of a human.

FIGS. 5A and 5B are diagrams showing the concept of detected regionsregarding a pupil, a face, and a body of a human.

FIGS. 6A and 6B are diagrams showing the concept of detected regionsregarding a pupil, a face, and a body of an animal.

FIGS. 7A and 7B are flowcharts illustrating AF operations in the digitalcamera.

FIG. 8 is a flowchart illustrating focus detection processing.

FIG. 9 is a flowchart illustrating the operations for selecting an AFmain frame.

FIG. 10 is a flowchart illustrating the operations for selecting an AFmain frame with priority on the detection center.

FIG. 11 is a flowchart illustrating the operations for selecting an AFmain frame with closest subject priority in a central region.

FIG. 12 is a flowchart illustrating the operations for selecting an AFmain frame with priority on the reliability degree of a detected region.

FIG. 13 is a flowchart illustrating the operations for selecting an AFmain frame with priority on the reliability degree of a central region.

FIG. 14 is a flowchart illustrating the operations for selecting an AFmain frame with priority on prediction in a central region.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made to an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

FIG. 1 is a block diagram showing a configuration of a digital camera300, which is one embodiment of an image capturing apparatus of thepresent invention.

In FIG. 1, the digital camera 300 is configured in such a manner that aninterchangeable lens 100 is detachably (interchangeably) attached to acamera main body 200 via a non-illustrated mount unit that includes anelectrical contact unit 106.

The interchangeable lens 100 includes a photographing lens 101 that hasa zoom mechanism as a photographing optical system, a diaphragm andshutter 102 for controlling a light amount, and a focusing lens 103 forachieving focus on an image sensor, which will be described later. Amotor 104 drives the focusing lens 103, and a lens controller 105controls the entirety of the interchangeable lens 100.

The camera main body 200 includes an image sensor 201. The image sensor201 includes a large number of pixels having photodiodes that convertreflected light from a subject into electrical signals. An A/Dconversion unit 202 includes a CDS circuit that removes output noise ofthe image sensor 201 and a nonlinear amplification circuit that operatesbefore A/D conversion, and converts analog signals from the image sensor201 into digital signals. The camera main body 200 further includes animage processing unit 203, an AF signal processing unit 204, a formatconversion unit 205, and a high-speed built-in memory (e.g., arandom-access memory; hereinafter referred to as a DRAM) 206.

An image recording unit 207 is composed of a recording medium, such as amemory card, and an interface therefor. A timing generator 208 generatestiming signals that control the timings of the operations of the digitalcamera 300, and a system control unit 209 controls the operations of theentirety of the digital camera, such as a shooting sequence. A lenscommunication unit 210 enables communication between the camera mainbody 200 and the interchangeable lens 100. A subject detection unit 211detects a subject from captured image signals.

An image display memory (hereinafter referred to as a VRAM) 212 storesimages for display. An image display unit 213 displays images, and alsoperforms display for assisting an operation and display of the state ofthe camera. The image display unit 213 further displays a shootingscreen and focus detection regions at the time of shooting. An operationunit 214 includes operation members for operating the camera fromoutside. A shooting mode switch 215 is an operation member for selectinga shooting mode, such as a macro mode and a sports mode. A main switch216 is a switch for turning ON the power of the digital camera 300. Whena release button is pressed halfway through, a release switch (SW1) 217is turned ON and causes shooting preparation operations, such as AF andAE, to start. When the release button is fully pressed, a release switch(SW2) 218 is turned ON and causes shooting operations to start.

The DRAM 206 is used as a high-speed buffer as temporary image storingmeans, a working memory for image compression and decompression, and thelike. The operation unit 214 includes various items. These various itemsinclude, for example, a menu switch for configuring various types ofsettings, such as settings of a shooting function and an imagereproduction function of the image capturing apparatus, an operationmode changeover switch for switching between a shooting mode and areproduction mode, and so forth.

The image sensor 201 is composed of a CCD or CMOS sensor. Light beamsthat have passed through the photographing optical system of theinterchangeable lens 100 are formed as an image on a light receivingsurface of the image sensor 201, and are converted by the photodiodesinto signal charges corresponding to the amount of incident light. Thesignal charges accumulated in each photodiode are converted into voltagesignals, and the voltage signals are read out sequentially from theimage sensor 201 based on driving pulses from the timing generator 208in accordance with an instruction from the system control unit 209.

Each pixel in the image sensor 201 used in the camera main body 200includes two (a pair of) photodiodes A, B, and one microlens that isprovided mutually for this pair of photodiodes A, B. Each pixel forms apair of optical images on the pair of photodiodes A, B by dividingincident light via the microlens, and outputs, from this pair ofphotodiodes A, B, a pair of pixel signals (an A signal and a B signal)to be used for an AF signal, will be described later. Also, an imagecapturing signal (an A+B signal) can be obtained by adding the outputsof the pair of photodiodes A, B.

Compositing together a plurality of A signals and a plurality of Bsignals output from a plurality of pixels will yield a pair of imagesignals as an AF signal (in other words, a focus detection signal) to beused in AF based on an image capturing surface phase-differencedetection method (hereinafter referred to as image capturing surfacephase-difference AF). The AF signal processing unit 204 calculates aphase difference (hereinafter referred to as an image displacementamount), which is the amount of displacement between this pair of imagesignals, by performing correlation computation with respect to this pairof image signals, and further calculates a defocus amount (as well as adefocus direction and reliability (a focus state)) of the photographingoptical system from this image displacement amount. Furthermore, it isassumed that the AF signal processing unit 204 calculates a defocusamount in a plurality of regions (focus detection regions) that can bedesignated on a screen.

The operations of the digital camera 300 of the present embodiment willbe described below using FIG. 2. FIG. 2 is a flowchart showing theoperations of the digital camera 300.

First, in step S201, the system control unit 209 checks the state of therelease switch (SW1) 217; the system control unit 209 proceeds to stepS202 when this release switch is ON, and stands by when this releaseswitch is OFF.

In step S202, the system control unit 209 sets AF frames (focusdetection frames), which will be described later, with respect to the AFsignal processing unit 204, and proceeds to step S203.

The system control unit 209 performs AF operations, which will bedescribed later, in step S203, and proceeds to step S204.

In step S204, the system control unit 209 checks the state of therelease switch (SW1) 217; processing proceeds to step S205 when thisrelease switch is ON, and returns to step S201 otherwise.

In step S205, the system control unit 209 checks the state of therelease switch (SW2) 218; processing proceeds to step S206 when thisrelease switch is ON, and returns to step S204 otherwise.

The system control unit 209 performs shooting operations in step S206,and then returns to step S201.

FIG. 3 is a flowchart for describing processing for setting focusdetection frames in the present embodiment.

First, in step S301, the system control unit 209 obtains subjectdetection information from the subject detection unit 211. In thepresent embodiment, it is assumed that a human or an animal, such as adog and a wild bird, as well as a main region inside this subject, isdetected as a subject. A main region denotes a pupil, a face, and a bodyof a human or an animal. As a method of detecting these, for example, alearning method based on deep learning or an image processing method,which are known techniques, are used. In the present embodiment, as aknown method can be used as a subject detection method, a detaileddescription of the subject detection method is omitted.

In step S302, the system control unit 209 judges whether a plurality ofmain regions were able to be detected from the result of detection bythe subject detection unit 211. When a plurality of main regions wereable to be detected, processing proceeds to step S303; otherwise,processing proceeds to step S304.

A description is now given of the concepts of detection for a case whereone main region was detected, and for a case where a plurality of mainregions were detected, using FIGS. 4A, 4B, 5A, and 5B. FIG. 4A shows astate where only a face a has been detected, whereas FIG. 5A shows astate where a pupil a, a face B, and a body C have been detected. It isassumed that a subject type, such as a human and an animal, and thecentral coordinates, the horizontal size, and the vertical size of eachdetected main region can be obtained from the subject detection unit211.

In step S303, the system control unit 209 inputs the smallest value ofthe sizes of the detected main regions, that is to say, the smaller oneof the values of the horizontal and vertical sizes of the region of thepupil A in FIG. 5A, as MinA, and regards MinA as one AF frame size.

In step S305, the system control unit 209 obtains a horizontal size Hshown in FIG. 5B, which includes all of the main regions, from thehorizontal coordinates and the horizontal size of each detected mainregion, and determines the number of AF frames in the horizontaldirection by dividing this H by the AF frame size MinA.

In step S307, the system control unit 209 obtains a vertical size Vshown in FIG. 5B, which includes all of the main regions, from thevertical coordinates and the vertical size of each detected main region,and determines the number of AF frames in the vertical direction bydividing this V by the AF frame size MinA. Then, the setting of AFframes is ended.

In the case of an animal as well, a control flow is similar to that forthe case of a human, and the concepts of detection regions and thesetting of AF frames are respectively as shown in FIG. 6A and FIG. 6B.Although the present embodiment incorporates square AF frames using thesmallest size of main regions, the AF frame size may vary between thehorizontal direction and the vertical direction, and the number of AFframes that can be computed by the system control unit 209 may be set.

In step S304, the system control unit 209 sets AF frames having apredetermined size X with respect to the detected face. As the size X, apupil size estimated from the face may be set, or a frame size that cansecure S/N and secure a sufficient focusing performance may be set inconsideration of a low-illuminance environment. It is assumed that theestimated pupil size is set as the size X in the present embodiment.

In step S306, the system control unit 209 sets the number Y of AF framesthat can include the region of the face a based on the AF frame size.

FIGS. 7A and 7B are flowcharts for describing the AF operations in stepS203 of FIG. 2.

First, the system control unit 209 detects a defocus amount andreliability (a reliability degree) by performing focus detectionprocessing in step S401, and proceeds to step S402. The focus detectionprocessing will be described later.

The system control unit 209 selects an AF main frame with use of thereliability degree obtained in step S401 in step S402, and proceeds tostep S403. The selection of the AF main frame will be described later.

In step S403, the system control unit 209 checks whether the reliabilitydegree of the defocus amount detected in step S401 is higher than areliability degree threshold 2 that has been set in advance. When thereliability degree is higher than the reliability degree threshold 2,processing proceeds to step S404; otherwise, processing proceeds to stepS413. Here, the reliability degree threshold 2 is set in such a mannerthat a reliability degree lower than the reliability degree threshold 2cannot guarantee the precision of the defocus amount, but can guaranteethe direction of the focus position of the subject. In step S404, thesystem control unit 209 checks whether the defocus amount detected instep S401 is equal to or smaller than a Def amount threshold 2 that hasbeen set in advance. When the defocus amount is equal to or smaller thanthe Def amount threshold 2, processing proceeds to step S405; otherwise,processing proceeds to step S412. Here, the Def amount threshold 2 isset in such a manner that a defocus amount equal to or smaller than theDef amount threshold 2 can control the focusing lens within the depth offocus if lens driving corresponding to the defocus amount is performedafterwards a predetermined number of times (e.g., three times) or less(e.g., an amount that is five times larger than the depth of focus isset as the Def amount threshold 2).

In step S405, the system control unit 209 checks whether the focusinglens 103 is in a stopped state; processing proceeds to step S406 whenthe focusing lens 103 is in the stopped state, and proceeds to step S410otherwise.

In step S406, the system control unit 209 checks whether the reliabilitydegree of the defocus amount detected in step S401 is higher than areliability degree threshold 1 that has been set in advance. When thereliability degree is higher than the reliability degree threshold 1,processing proceeds to step S407; otherwise, processing proceeds to stepS410. Here, the reliability degree threshold 1 is set in such a mannerthat a reliability degree higher than the reliability degree threshold 1keeps variations in the precision of the defocus amount within apredetermined range (e.g., within the depth of focus).

In step S407, the system control unit 209 checks whether the defocusamount detected in step S401 is equal to or smaller than a Def amountthreshold 1 that has been set in advance. When the defocus amount isequal to or smaller than the Def amount threshold 1, processing proceedsto step S408; otherwise, processing proceeds to step S409. Here, the Defamount threshold 1 is set in such a manner that a detected defocusamount equal to or smaller than the Def amount threshold 1 causes thefocusing lens to be controlled within the depth of focus.

In step S408, the system control unit 209 determines that the currentstate is an in-focus state, and ends the present flow.

In step S409, the system control unit 209 drives the focusing lens 103by the defocus amount detected in step S401, and then returns to stepS401.

By performing the sequence of operations from step S405 to step S409,the defocus amount can be detected again in a state where the lens hasstopped in a case where the reliability degree detected in step S401 ishigher than the reliability degree threshold 1.

In step S410, the system control unit 209 drives the focusing lens 103by a predetermined percentage of the defocus amount detected in stepS401, and proceeds to step S411.

In step S411, the system control unit 209 issues an instruction forstopping the focusing lens 103, and returns to step S401.

In step S412, the system control unit 209 drives the focusing lens 103by a predetermined percentage of the defocus amount detected in stepS401, and returns to step S401. Here, the predetermined percentage isset so that the amount of lens driving is smaller than the defocusamount (e.g., 80 percent). Furthermore, the lens speed is set so that itis, for example, slower than the speed at which lens driving can beperformed exactly in one frame period. This can prevent surpassing ofthe focus position of the subject when the detected defocus amount isnot accurate, and can further perform the next lens driving whiledriving the lens without stopping the lens (overlap control).

In step S413, the system control unit 209 checks whether an out-of-focuscondition has been satisfied. When the out-of-focus condition has beensatisfied, processing proceeds to step S414; otherwise, processingproceeds to step S415. Here, the out-of-focus condition is a conditionin which it is determined that there is no subject to be focused on, andis, for example, a case where lens driving has been completed throughoutthe movable range of the focusing lens 103. That is to say, it is acondition in which the focusing lens 103 has returned to the initialposition after detecting both of the lens ends on the far side and thenear side.

In step S414, the system control unit 209 determines that the currentstate is an out-of-focus state, and ends the present flow.

In step S415, the system control unit 209 checks whether the focusinglens 103 has reached the lens end on the far side or the near side. Whenthe lens end has been reached, processing proceeds to step S416;otherwise, processing proceeds to step S417.

In step S416, the system control unit 209 reverses the direction ofdriving of the focusing lens 103, and returns to step S401. In stepS417, the focusing lens 103 is driven in a predetermined direction, andprocessing returns to step S401. The focusing lens speed is set at, forexample, the highest speed within the lens speed range in which thefocus position is not surpassed when the defocus amount can be detected.

The focus detection processing in step S401 will be described using FIG.8.

First, in step S501, the system control unit 209 sets a focus detectionregion having an arbitrary range inside the image sensor 201, andproceeds to step S502.

In step S502, the system control unit 209 obtains pairs of image signals(an A image and a B image) for focus detection from the image sensor 201with respect to the focus detection region set in step S501, andproceeds to step S503.

In step S503, the system control unit 209 performs processing for addingand averaging the pairs of signals obtained in step S502 in the verticaldirection, and then proceeds to step S504. This processing can alleviatethe influence of noise of the image signals.

In step S504, the system control unit 209 performs filter processing forextracting signal components in a predetermined frequency band from theresult of adding and averaging the signals in the vertical direction instep S503, and then proceeds to step S505.

In step S505, the system control unit 209 calculates a correlationamount from the signals resulting from the filter processing in stepS504, and proceeds to step S506.

In step S506, the system control unit 209 calculates a correlationchange amount from the correlation amount calculated in step S505, andproceeds to step S507.

In step S507, the system control unit 209 calculates an imagedisplacement amount from the correlation change amount calculated instep S506, and proceeds to step S508.

In step S508, the system control unit 209 calculates a reliabilitydegree indicating the extent to which the image displacement amountcalculated in step S507 can be trusted, and proceeds to step S509.

In step S509, the system control unit 209 converts the imagedisplacement amount into a defocus amount, and ends the focus detectionprocessing.

FIG. 9 is a flowchart illustrating the operations for selecting a mainframe in step S402 of FIG. 7A. In selecting a main focus detectionframe, the main focus detection frame is searched for in order from asubject with a high priority degree depending on parts of the subject.

First, in step S601, the system control unit 209 sets a main frame at aninitial position as a preliminary preparation for the main frameselection.

In step S602, the system control unit 209 judges whether the subjectdetection unit 211 has detected a pupil of a subject. When the pupil hasbeen detected, processing proceeds to step S604; when the pupil has notbeen detected, processing proceeds to step S603.

The system control unit 209 sets a pupil region as a main frameselection region in step S604, and performs the main frame selectionwith priority on the detection center, which will be described later, inthe next step S605.

In step S607, the system control unit 209 judges whether the reliabilitydegree of the selected main frame is equal to or higher than a mainframe reliability degree threshold with respect to the result ofselection of the main frame in the pupil region. When the reliabilitydegree is judged to be equal to or higher than the threshold, processingproceeds to step S614; otherwise, processing proceeds to step S603. StepS607 is for determining whether the detected defocus amount of the mainframe selected in the detected region of the pupil is equal to orsmaller than a predetermined variation. For example, the aforementionedreliability degree threshold 1 or the like may be set as the main framereliability degree threshold. When it is determined that it is difficultto select the main frame in the pupil region in step S607, processingproceeds to step S603 to select the main frame in a face region.

In step S603, the system control unit 209 judges whether the subjectdetection unit 211 has detected a face of the subject. When the face hasbeen detected, processing proceeds to step S608; when the face has notbeen detected, processing proceeds to step S611.

The system control unit 209 sets a face region as the main frameselection region in step S608, and performs the main frame selectionwith priority on the detection center, which will be described later, inthe next step S609.

In step S610, the system control unit 209 judges whether the reliabilitydegree of the selected frame is equal to or higher than the main framereliability degree threshold with respect to the result of selection ofthe main frame in the face region. When the reliability degree is judgedto be equal to or higher than the threshold, processing proceeds to stepS614; otherwise, processing proceeds to step S611. Step S610 is fordetermining whether the detected defocus amount of the main frameselected in the detected region of the face is equal to or smaller thana predetermined variation. For example, the reliability degree threshold1 or the like may be set as the main frame reliability degree threshold,similarly to the case of the pupil region. When it is determined that itis difficult to select the main frame in the face region in step S610,processing proceeds to step S611 to select the main frame in a bodyregion.

In step S611, the system control unit 209 judges whether the subjectdetection unit 211 has detected a body of the subject. When the body hasbeen detected, processing proceeds to step S612; when the body has notbeen detected, processing proceeds to step S614.

The system control unit 209 sets a body region as the main frameselection region in step S612, and performs the main frame selectionwith closest subject priority in a central region, which will bedescribed later, in the next step S613.

In step S614, the system control unit 209 judges whether the main frameis at the initial position in order to check whether the main frame wasultimately able to be set on one of the pupil, the face, and the body.When the main frame corresponds to an initial value, processing proceedsto step S615; otherwise, the main frame selection processing is ended.

In step S615, the system control unit 209 selects main frames atmultiple points. Note that in step S615, while it is possible to adopt amethod of, for example, selecting the main frame in a predeterminedregion inside the screen without using detection information, as this isnot a main portion of the present embodiment, a detailed descriptionthereof is omitted.

Next, a description is given of main frame selection processingappropriate for each detected part with use of FIG. 10 and FIG. 11. Inthe present embodiment, when the detected part is a pupil or a face, thedetected part takes up a relatively small region inside the subject, andthus main frame selection processing that gives importance to theposition of the detection center is used. FIG. 10 is a flowchart of theoperations for selecting the main frame with priority on the detectioncenter.

From step S701 to step S704, similar processing is executed with respectto every focus detection frame inside the focus detection region.

In step S702, the system control unit 209 judges whether the focusdetection frame is closer to the center of the detection region than thefocus detection frame currently set as the main frame. When the focusdetection frame is closer, processing proceeds to step S703 to updatethe main frame; when the focus detection frame is farther, processingproceeds to step S704. The foregoing processing is executed with respectto every focus detection frame, and the main frame selection processingis completed.

Next, a description is given of main frame selection processing for acase where the detected part is a body. When the detected part is abody, a relatively large region is detected inside the subject. Then, inselecting the main frame in a body region that has a possibility ofhaving a complex subject shape, main frame selection processing isperformed while giving importance to a frame in which the subject is notabsent in a central region with a high probability of the existence ofthe subject inside the detected region (closest subject priority). FIG.11 is a flowchart showing the operations for selecting the main framewith closest subject priority in the central region.

In step S801, the system control unit 209 configures the initial settingof the number of frames for setting a main frame search region in thenext step S802.

The system control unit 209 sets the main frame search region in stepS802, and proceeds to step S803. From step S803 to step S807, similarprocessing is executed with respect to every focus detection frameinside the focus detection region.

In step S804, the system control unit 209 determines whether the focusdetection frame is inside the main frame search region; when the focusdetection frame is inside the main frame search region, processingproceeds to step S805. In step S805, the system control unit 209determines whether the defocus amount of the focus detection framecorresponds to a shorter distance than the detected defocus amount ofthe main frame currently set. When the defocus amount of the focusdetection frame corresponds to a shorter distance, the main frame isupdated in the next step S806; when the defocus amount of the focusdetection frame does not correspond to a shorter distance, processingproceeds to step S807, thereby proceeding to the determination about thenext focus detection frame. The foregoing processing is executed withrespect to every focus detection frame, and processing proceeds to stepS808.

In step S808, the system control unit 209 judges whether the reliabilitydegree of the detected defocus amount of the selected main frame isequal to or higher than a predetermined threshold. When the reliabilitydegree is judged to be equal to or higher than the predeterminedthreshold, the main frame selection processing is completed. When thereliability degree is judged to be lower than the predeterminedthreshold in step S808, processing proceeds to steps S809 and S810, themain frame search region is enlarged, and the aforementioned steps S802to S808 are executed. When the main frame search has been completed inthe entire focus detection region in step S809, the main frame selectionprocessing is completed.

Application of the present embodiment enables the execution of accuratefocus adjustment in a region with a higher priority degree whileavoiding regions in which focus detection is difficult in a case where aplurality of subject parts have been detected. For example, in a casewhere a subject is an animal that moves around against thephotographer's intention, such as a dog, a cat, and a bird, focusadjustment can be performed in a part with a higher priority degreewhile avoiding parts in which it is difficult to perform focus detectiondue to a low reliability degree of the detected defocus amount.

Although one embodiment of the present invention has been described indetail thus far, the present invention is not limited to the foregoingembodiment, and various embodiments that do not depart from theprinciples of this invention are also encompassed within the presentinvention.

In the present embodiment, when a pupil or a face has been detected,main frame selection processing that gives importance to the position ofthe detection center is used, whereas when a body has been detected,main frame selection processing that gives importance to a frame inwhich the subject is not absent in a central region with a highprobability of the existence of the subject inside the detected regionis used. With respect to a subject that tends to have a relatively smallsize inside the subject, such as a pupil and a face, main frameselection processing that gives importance to the position of thedetection center is executed under the assumption that there is almostno displacement in the detected position of the subject. However, forexample, when the displacement in the detected position needs to betaken into consideration, processing for selecting a focus detectionframe that has a detected focus amount with a higher reliability degreeas the main frame may be performed. FIG. 12 is a flowchart showing theoperations for selecting the main frame with priority on a focusdetection frame that has a detected defocus amount with a highreliability degree inside the detected region.

From step S901 to step S905, similar processing is executed with respectto every focus detection frame inside the focus detection region. Instep S902, the system control unit 209 judges whether the focusdetection frame is inside the region of the detected part of thesubject. When the focus detection frame is inside the region, whetherthe focus detection frame has higher reliability than the focusdetection frame currently set as the main frame is judged in the nextstep S903; when the focus detection frame has higher reliability,processing proceeds to step S904 to update the main frame. When it isjudged that the focus detection frame has lower reliability than thefocus detection frame currently set as the main frame in step S903,processing proceeds to step S905, this determination is made withrespect to every focus detection frame, and the main frame selectionprocessing is completed.

Furthermore, in the present embodiment, in a case where a body has beendetected and the main frame has a low reliability degree on a pupil or aface, main frame selection processing that gives importance to a framein which the subject is not absent in a central region with a highprobability of the existence of the subject is used with respect to theregion of the body in consideration of the degree of complexity of thesubject shape. However, for example, when the degree of complexity ofthe subject shape need not be taken into consideration, processing forselecting a focus detection frame that has a detected defocus amountwith a higher reliability degree as a main frame may be performed. FIG.13 is a flowchart showing the operations for selecting the main framewith priority on the reliability degree of the central region.

In step S1001, the system control unit 209 configures the initialsetting of the number of frames for setting a main frame search regionin the next step S1002.

The system control unit 209 sets the main frame search region in stepS1002, and proceeds to step S1003. From step S1003 to step S1007,similar processing is executed with respect to every focus detectionframe inside the focus detection region.

In step S1004, the system control unit 209 determines whether the focusdetection frame is inside the main frame search region; when the focusdetection frame is inside the main frame search region, processingproceeds to step S1005. In step S1005, whether the focus detection framehas the detected defocus amount with a higher reliability degree thanthe main frame currently set is determined. When the focus detectionframe has the detected defocus amount with a higher reliability degree,the main frame is updated in the next step S1006; when the focusdetection frame has the detected defocus amount with a lower reliabilitydegree, processing proceeds to S1007, thereby proceeding to thedetermination about the next focus detection frame. The foregoingprocessing is executed with respect to every focus detection frame, andprocessing proceeds to step S1008.

In step S1008, the system control unit 209 judges whether thereliability degree of the detected defocus amount of the selected mainframe is equal to or higher than a predetermined threshold. When thereliability degree is judged to be equal to or higher than thepredetermined threshold in step S1008, the main frame selectionprocessing is completed. When the reliability degree is judged to belower in step S1008, processing proceeds to steps S1009 and S1010, themain frame search region is enlarged, and the aforementioned steps S1002to S1008 are executed. When the main frame search has been completed inthe entire focus detection region in step S1009, the main frameselection processing is completed.

Furthermore, in a scene that has a high risk of, for example, absence ofthe subject irrespective of the detected part of the subject, such as acase where the motion of the subject is strenuous, main frame selectionprocessing may be set based on the determination about whether a focusadjustment mode is a mode for shooting of a motionless object (amotionless subject) or a mode for shooting of a dynamic object (a movingsubject). In the case of the mode for shooting of a dynamic object, thefocus position is tracked by predicting the subject position in a targetframe from history information of the subject positions in a pluralityof frames from the past. The present modification example will bedescribed using processing for preferentially selecting, as a mainframe, a focus detection frame indicating a position that is close tothe aforementioned predicted subject position. FIG. 14 is a flowchartshowing the operations for selecting the main frame with priority onprediction in the central region.

In step S1101, the system control unit 209 configures the initialsetting of the number of frames for setting a main frame search regionin the next step S1102. The system control unit 209 sets the main framesearch region in step S1102, and proceeds to step S1103.

From step S1103 to step S1107, similar processing is executed withrespect to every focus detection frame inside the focus detectionregion. In step S1104, the system control unit 209 determines whetherthe focus detection frame is inside the main frame search region; whenthe focus detection frame is inside the main frame search region,processing proceeds to step S1105. In step S1105, whether the focusdetection frame is closer than the main frame currently set to thesubject position predicted from the detected defocus amount and thecurrent lens position is determined. When the focus detection frame iscloser, the main frame is updated in the next step S1106; when the focusdetection frame is farther, processing proceeds to S1107, therebyproceeding to the determination about the next focus detection frame.The foregoing processing is executed with respect to every focusdetection frame, and processing proceeds to step S1108.

In step S1108, the system control unit 209 judges whether thereliability degree of the detected defocus amount of the selected mainframe is equal to or higher than a predetermined threshold. When thereliability degree is judged to be equal to or higher than thepredetermined threshold in step S1108, the main frame selectionprocessing is completed. When the reliability degree is judged to belower in step S1108, processing proceeds to steps S1109 and S1110, themain frame search region is enlarged, and the aforementioned steps S1102to S1108 are executed. When the main frame search has been completed inthe entire focus detection region in step S1109, the main frameselection processing is completed.

Application of the present modification example enables the execution ofaccurate focus adjustment in a region with a higher priority degree,also with respect to a subject with strenuous motion, while avoidingregions in which focus detection is difficult.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toembodiments, it is to be understood that the invention is not limited tothe disclosed embodiments but is defined by the scope of the followingclaims.

This application claims the benefit of Japanese Patent Application No.2020-075608, filed Apr. 21, 2020 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus, comprising: animage sensor configured to capture an image of a subject; at least oneprocessor or circuit configured to function as a detection unitconfigured to detect the subject from the image captured by the imagesensor; a setting unit configured to set a plurality of focus detectionframes inside the image based on a result of the detection by thedetection unit; a focus detection unit configured to detect a focusstate and reliability thereof in each of the plurality of focusdetection frames; and a selection unit configured to select a main focusdetection frame for performing focus adjustment based on results of thedetections by the detection unit and the focus detection unit, whereinthe selection unit is configured to cause a method of selecting the mainfocus detection frame to vary depending on a part of the subjectdetected by the detection unit, and wherein the selection unit isconfigured to cause the method of selecting the main focus detectionframe to vary between a case where the subject is a motionless subjectand a case where the subject is a moving subject.
 2. The image capturingapparatus according to claim 1, wherein the selection unit is configuredto cause the method of selecting the main focus detection frame to varybetween a case where a focus adjustment mode is a mode for focusing on amotionless subject and a case where the focus adjustment mode is a modefor focusing on a moving subject.
 3. The image capturing apparatusaccording to claim 1, wherein the selection unit is configured to causethe method of selecting the main focus detection frame to vary dependingon a size of the part detected by the detection unit.
 4. The imagecapturing apparatus according to claim 1, wherein the selection unit isconfigured to select the main focus detection frame based on the resultof the detection by the detection unit, the setting of the setting unit,and the result of the detection by the focus detection unit.
 5. Theimage capturing apparatus according to claim 1, wherein the setting unitis configured to, when the detection unit has detected a plurality ofparts of the subject corresponding to a plurality of detected regions,set the plurality of focus detection frames with respect to a regionthat includes the plurality of detected regions.
 6. The image capturingapparatus according to claim 5, wherein the setting unit is configuredto set the plurality of focus detection frames based on a size of thesmallest one of the detected regions corresponding to the parts of thesubjects detected by the detection unit.
 7. The image capturingapparatus according to claim 1, wherein the selection unit is configuredto, when a pupil has been detected as the part of the subject, selectthe main focus detection frame with priority on a center of the part. 8.The image capturing apparatus according to claim 1, wherein theselection unit is configured to, when a face has been detected as thepart of the subject, select the main focus detection frame with priorityon a center of the part.
 9. The image capturing apparatus according toclaim 1, wherein the selection unit is configured to, when a body hasbeen detected as the part of the subject, select the main focusdetection frame with priority on a closest subject.
 10. A control methodfor an image capturing apparatus that includes image sensor forcapturing an image of a subject, the control method comprising:detecting the subject from the image captured by the image sensor;setting a plurality of focus detection frames inside the image based ona result of the detection in the detecting; performing focus detectionto detect a focus state and reliability thereof in each of the pluralityof focus detection frames; and selecting a main focus detection framefor performing focus adjustment based on results of the detections inthe detecting and the focus detection, wherein the selecting causes amethod of selecting the main focus detection frame to vary depending ona part of the subject detected in the detecting, and wherein theselecting causes the method of selecting the main focus detection frameto vary between a case where the subject is a motionless subject and acase where the subject is a moving subject.
 11. A non-transitorycomputer-readable storage medium that stores a program for causing acomputer to execute each step of a control method for an image capturingapparatus that includes image sensor for capturing an image of asubject, the control method comprising: detecting the subject from theimage captured by the image sensor; setting a plurality of focusdetection frames inside the image based on a result of the detection inthe detecting; performing focus detection to detect a focus state andreliability thereof in each of the plurality of focus detection frames;and selecting a main focus detection frame for performing focusadjustment based on results of the detections in the detecting and thefocus detection, wherein the selecting causes a method of selecting themain focus detection frame to vary depending on a part of the subjectdetected in the detecting, and wherein the selecting causes the methodof selecting the main focus detection frame to vary between a case wherethe subject is a motionless subject and a case where the subject is amoving subject.
 12. The non-transitory computer-readable storage mediumaccording to claim 11, wherein the selecting causes the method ofselecting: (a) the main focus detection frame to vary between a casewhere a focus adjustment mode is a mode for focusing on a motionlesssubject and a case where the focus adjustment mode is a mode forfocusing on a moving subject or (b) the main focus detection frame tovary depending on a size of the part detected by the detection unit. 13.The non-transitory computer-readable storage medium according to claim11, wherein the selecting selects the main focus detection frame basedon the result of the detection by the detecting, the setting, and theresult of the detection by the focus detecting.
 14. The non-transitorycomputer-readable storage medium according to claim 11, wherein thesetting, when the detecting has detected a plurality of parts of thesubject corresponding to a plurality of detected regions, sets theplurality of focus detection frames with respect to a region thatincludes the plurality of detected regions.
 15. The non-transitorycomputer-readable storage medium according to claim 11, wherein theselecting, when a pupil has been detected as the part of the subject,selects the main focus detection frame with priority on a center of thepart.
 16. The control method according to claim 10, wherein theselecting causes the method of selecting: (a) the main focus detectionframe to vary between a case where a focus adjustment mode is a mode forfocusing on a motionless subject and a case where the focus adjustmentmode is a mode for focusing on a moving subject or (b) the main focusdetection frame to vary depending on a size of the part detected by thedetection unit.
 17. The control method according to claim 10, whereinthe selecting selects the main focus detection frame based on the resultof the detection by the detecting, the setting, and the result of thedetection by the focus detecting.
 18. The control method according toclaim 10, wherein the setting, when the detecting has detected aplurality of parts of the subject corresponding to a plurality ofdetected regions, sets the plurality of focus detection frames withrespect to a region that includes the plurality of detected regions. 19.The control method according to claim 10, wherein the selecting, when apupil has been detected as the part of the subject, selects the mainfocus detection frame with priority on a center of the part.